feat(ensindexer): introduce Indexing Status Builder module#1613
Conversation
This module aims to host an abstraction layer that primarly relies on `viem` and `@ensnode/ponder-sdk` abstractions. It is still using `@ensnode/ensnode-sdk` for convenience, but leave the possiblity to iterate and remove `@ensnode/ensnode-sdk` dependency in the future.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 3 Skipped Deployments
|
|
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe PR adds block comparison utilities to ponder-sdk and introduces an indexing-status-builder module in ensindexer that constructs per-chain and omnichain indexing status snapshots from metadata, with new types for backfill scope and chain indexing metadata. A workspace dependency on ponder-sdk is added. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Introduces a new “Indexing Status Builder” module in apps/ensindexer intended to sit between ENSIndexer and @ensnode/ponder-sdk/viem, and updates @ensnode/ponder-sdk’s indexing-status data model to support that abstraction.
Changes:
- Add
@ensnode/ponder-sdkas a dependency ofapps/ensindexer. - Extend
ponder-sdktypes/APIs (exports,PonderIndexingStatusshape, block helpers/types). - Add initial (WIP) builder/validation scaffolding for producing
OmnichainIndexingStatusSnapshotfrom Ponder metrics/status.
Reviewed changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| pnpm-lock.yaml | Adds workspace link for @ensnode/ponder-sdk under apps/ensindexer. |
| apps/ensindexer/package.json | Declares @ensnode/ponder-sdk dependency. |
| packages/ponder-sdk/src/indexing-status.ts | Changes chains map value to ChainIndexingStatus (with checkpointBlock). |
| packages/ponder-sdk/src/deserialize/indexing-status.ts | Updates deserialization to build ChainIndexingStatus objects. |
| packages/ponder-sdk/src/index.ts | Re-exports blocks and chains from the SDK entrypoint. |
| packages/ponder-sdk/src/blocks.ts | Adds block comparison helpers and blockrange types. |
| apps/ensindexer/src/lib/indexing-status-builder/chain-block-refs.ts | New logic to fetch/configure per-chain BlockRefs via viem. |
| apps/ensindexer/src/lib/indexing-status-builder/chain-indexing-status-snapshot.ts | New per-chain snapshot derivation based on metrics/status/config. |
| apps/ensindexer/src/lib/indexing-status-builder/omnichain-indexing-status-snapshot.ts | New omnichain snapshot builder using ensnode-sdk omnichain helpers. |
| apps/ensindexer/src/lib/indexing-status-builder/validate/*.ts | Placeholder “validate” functions for chain/omnichain snapshots. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
apps/ensindexer/src/lib/indexing-status-builder/chain-indexing-status-snapshot.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/validate/chain-indexing-status-snapshot.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/validate/omnichain-indexing-status-snapshot.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/omnichain-indexing-status-snapshot.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-block-refs.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-block-refs.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-indexing-status-snapshot.ts
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 10
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/ponder-sdk/src/indexing-status.ts (1)
27-29:⚠️ Potential issue | 🟡 MinorStale JSDoc: doc still says "block reference" but the value type is now
ChainIndexingStatus.Line 28 reads "Map of indexed chain IDs to their block reference" but the value type was changed to
ChainIndexingStatus. Consider updating to reflect the new structure.📝 Suggested fix
/** - * Map of indexed chain IDs to their block reference. + * Map of indexed chain IDs to their chain indexing status. * * Guarantees: * - Includes entry for at least one indexed chain. */
🤖 Fix all issues with AI agents
In `@apps/ensindexer/src/lib/indexing-status-builder/chain-block-refs.ts`:
- Around line 14-28: The null-check on the result of publicClient.getBlock in
fetchBlockRef is dead code because viem throws on missing blocks; remove the
unreachable if (!block) branch and instead ensure you never pass null into
bigIntToNumber by asserting or guarding block.number and block.timestamp before
calling bigIntToNumber (e.g., use non-null assertions or explicit checks) so
deserializeBlockRef receives valid numbers; keep the call to
publicClient.getBlock as-is and update fetchBlockRef to rely on viem errors
while preventing block.number/block.timestamp from being null when calling
bigIntToNumber/deserializeBlockRef.
- Around line 108-110: The catch block currently swallows the original exception
and throws a generic Error; update it to preserve the original error as the
cause so debugging info is retained: capture the caught error (e.g., catch
(err)) and rethrow using the Error cause option or include err.message in the
new Error (for example: throw new Error(`Could not get BlockRefs for chain
${chainId}`, { cause: err }) ), referencing the catch in chain-block-refs.ts
where the current throw occurs and the chainId variable used in the message.
- Around line 64-111: The loop over chainIds is performing RPCs serially; change
it to run per-chain work in parallel by mapping chainIds to an array of async
tasks and awaiting Promise.all (or Promise.allSettled with error aggregation).
For each chainId task, read chainsConfigBlockrange, publicClients, and
chainsIndexingMetrics, compute startBlock/endBlock/backfillEndBlock, call
fetchBlockRef for the three refs, construct the ChainBlockRefs object, and
return [chainId, chainBlockRef]; after Promise.all resolve, populate
chainsBlockRefs from the results (or throw a combined error if any task failed).
Ensure you keep the same validation checks
(startBlock/publicClient/indexingMetrics/historicalTotalBlocks) inside each task
and include chainId in any thrown errors to aid debugging.
In
`@apps/ensindexer/src/lib/indexing-status-builder/chain-indexing-status-snapshot.ts`:
- Around line 46-94: The Queued branch (when
isBlockRefEqualTo(chainBlockRefs.config.startBlock, checkpointBlock) is true)
can mask contradictory states if chainIndexingMetrics.indexingCompleted or
indexingRealtime are also true; update the queued branch in
chain-indexing-status-snapshot to emit a defensive warning (e.g., logger.warn)
including chainId, checkpointBlock and which metric(s) are set before returning
the Queued snapshot so these inconsistencies are visible in logs; keep the
existing return value (validateChainIndexingStatusSnapshot with
ChainIndexingStatusIds.Queued) but do not change downstream logic.
In
`@apps/ensindexer/src/lib/indexing-status-builder/omnichain-indexing-status-snapshot.ts`:
- Around line 33-34: The code currently calls getOmnichainIndexingStatus(chains)
and then calls getOmnichainIndexingCursor(chains), but
getOmnichainIndexingCursor internally recomputes getOmnichainIndexingStatus —
avoid the double computation by computing omnichainStatus once and passing it
into the cursor helper (or refactor getOmnichainIndexingCursor to accept an
already-computed status): change the call to something like
getOmnichainIndexingCursor(chains, omnichainStatus) and update the helper
function signature (and any other call sites) so it uses the provided
omnichainStatus instead of calling getOmnichainIndexingStatus again; ensure
tests/usage are updated accordingly.
- Around line 36-67: The switch on omnichainStatus (in
omnichain-indexing-status-snapshot.ts) is missing a default/exhaustiveness guard
so new OmnichainIndexingStatusIds variants could cause an undefined return; add
a final default that throws an informative error (or call an assertUnreachable
utility) including the omnichainStatus value to ensure exhaustiveness and
preserve the function's return contract for
validateOmnichainIndexingStatusSnapshot callers.
In
`@apps/ensindexer/src/lib/indexing-status-builder/validate/chain-indexing-status-snapshot.ts`:
- Around line 1-7: validateChainIndexingStatusSnapshot currently returns its
input without checks; replace the no-op with real validation of the
discriminated union ChainIndexingStatusSnapshot by switching on the discriminant
(the variant tag) and asserting each variant's required fields and types (e.g.,
Queued, Backfill, Following, Completed-specific invariants such as required
numeric offsets, timestamps, or nullable fields), throwing a descriptive Error
on failure; if you cannot implement full checks now, replace the body with a
clear TODO comment mentioning validateChainIndexingStatusSnapshot and
short-circuit with a runtime assertion (e.g., throw new Error("TODO:
validateChainIndexingStatusSnapshot not implemented")) so callers don't get
silent false confidence.
In
`@apps/ensindexer/src/lib/indexing-status-builder/validate/omnichain-indexing-status-snapshot.ts`:
- Around line 3-7: The current validateOmnichainIndexingStatusSnapshot function
is a no-op leaving unsafe `as` casts unchecked; update
validateOmnichainIndexingStatusSnapshot to add a TODO with a tracking
ticket/reference and implement minimal runtime guards: assert the top-level
discriminant (omnichainStatus) and switch over its known cases (use an
assertNever fallback) and for each entry validate the per-chain `chainStatus`
discriminant and expected shape (e.g., verify Maps contain expected value types)
so the identity cast is safe at runtime; reference the function name
validateOmnichainIndexingStatusSnapshot, the omnichainStatus discriminant,
chainStatus, and use an assertNever-style helper for unhandled cases.
In `@packages/ponder-sdk/src/blocks.ts`:
- Around line 33-51: The ordering function isBlockRefBefore currently requires
both number and timestamp to be strictly less which misorders blocks when
timestamps are equal; change isBlockRefBefore to compare only blockA.number <
blockB.number (use block number as canonical ordering), update
isBlockRefBeforeOrEqualTo to rely on that (or number ===), and optionally add a
separate validation/assertion (e.g., inside isBlockRefEqualTo or a new
validateBlockRefConsistency helper) to log or throw when block numbers are equal
but timestamps differ so timestamp mismatches are surfaced without affecting
ordering.
- Around line 74-94: BlockrangeWithStartBlock is unused internally but is
re-exported publicly via index.ts; mark it as deprecated to avoid breaking
consumers now and communicate planned removal in the next major release. Add a
JSDoc `@deprecated` tag to the BlockrangeWithStartBlock interface with a short
message like "Unused internally; scheduled for removal in next major release"
(leave the export in index.ts unchanged so consumers keep compatibility) and
update any public docs/changelog to note the deprecation.
apps/ensindexer/src/lib/indexing-status-builder/chain-block-refs.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-block-refs.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-block-refs.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-indexing-status-snapshot.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/omnichain-indexing-status-snapshot.ts
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/omnichain-indexing-status-snapshot.ts
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/validate/chain-indexing-status-snapshot.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/validate/omnichain-indexing-status-snapshot.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/cross-chain-indexing-status-snapshot.ts
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-indexing-status-snapshot.ts
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Fix all issues with AI agents
In `@apps/ensindexer/src/lib/indexing-status-builder/chain-block-refs.ts`:
- Around line 90-96: The code currently only checks that
indexingMetrics.backfillSyncBlocksTotal is a number; update validation so
historicalTotalBlocks (the value assigned from
indexingMetrics.backfillSyncBlocksTotal) is a positive integer (use
Number.isInteger and > 0) before computing backfillEndBlock = startBlock +
historicalTotalBlocks - 1; if the check fails throw an Error that includes
chainId and the invalid value to make the misconfiguration obvious.
- Around line 31-33: The catch block in chain-block-refs.ts currently creates a
new Error with the message but drops the original error; update the throw to
preserve the original error via the Error cause option (i.e., throw new
Error(`Failed to fetch block ref for block number ${blockNumber}:
${errorMessage}`, { cause: error })) so the original stack and identity are
retained for proper error chaining; keep the existing errorMessage logic (error
instanceof Error ? error.message : "Unknown error") and pass the original error
variable as the cause when constructing the new Error.
In
`@apps/ensindexer/src/lib/indexing-status-builder/corss-chain-indexing-status-snapshot.ts`:
- Around line 1-6: The file name has a typo
("corss-chain-indexing-status-snapshot.ts"); rename it to
"cross-chain-indexing-status-snapshot.ts" and update every import/reference that
points to the old name so modules can resolve correctly; check for usages
referencing CrossChainIndexingStatusSnapshotOmnichain,
CrossChainIndexingStrategyIds, and OmnichainIndexingStatusSnapshot and adjust
their import paths to the new file name, then run TypeScript/IDE imports fix to
ensure no broken references remain.
In `@packages/ponder-sdk/src/index.ts`:
- Around line 6-7: The top-level exports of numbers and time in
packages/ponder-sdk/src/index.ts surface a zod/v4 dependency via exported
validators (symbols: numbers, time, and zod) which may violate the Layer 1
constraint; decide whether zod is permitted and then either (A) keep the exports
but document/accept the zod dependency, or (B) refactor so zod types/validators
are internal only: remove re-exports from the public index.ts and instead export
plain JS/TS types or helper functions (or convert validators to simple runtime
checks) so no zod symbols leak from numbers.ts/time.ts; update the module
exports in index.ts and adjust numbers.ts/time.ts to either internalize zod
imports or expose non-zod-safe public APIs accordingly.
apps/ensindexer/src/lib/indexing-status-builder/chain-block-refs.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-block-refs.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/cross-chain-indexing-status-snapshot.ts
Show resolved
Hide resolved
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 10 changed files in this pull request and generated 9 comments.
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
apps/ensindexer/src/lib/indexing-status-builder/chain-block-refs.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-block-refs.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/cross-chain-indexing-status-snapshot.ts
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/validate/chain-indexing-status-snapshot.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-indexing-status-snapshot.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-indexing-status-snapshot.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-block-refs.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-block-refs.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-indexing-status-snapshot.ts
Outdated
Show resolved
Hide resolved
|
@greptile review |
ffe498d to
7516a25
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 14 changed files in this pull request and generated 2 comments.
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
apps/ensindexer/src/lib/indexing-status-builder/chain-indexing-status-snapshot.test.ts
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/omnichain-indexing-status-snapshot.ts
Outdated
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-indexing-status-snapshot.test.ts
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/chain-indexing-metadata.mock.ts
Outdated
Show resolved
Hide resolved
7516a25 to
34d7920
Compare
|
@greptile review |
apps/ensindexer/src/lib/indexing-status-builder/omnichain-indexing-status-snapshot.ts
Show resolved
Hide resolved
apps/ensindexer/src/lib/indexing-status-builder/omnichain-indexing-status-snapshot.ts
Show resolved
Hide resolved
|
@greptile review |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 14 changed files in this pull request and generated 4 comments.
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
apps/ensindexer/src/lib/indexing-status-builder/omnichain-indexing-status-snapshot.ts
Show resolved
Hide resolved
|
@greptile review |
lightwalker-eth
left a comment
There was a problem hiding this comment.
@tk-o Looks good, thank you 👍 Please merge when ready!
| * The backfill phase progresses from `startBlock` toward `endBlock`, and ends | ||
| * when the indexed block reaches `endBlock`. | ||
| */ | ||
| export interface BackfillScope { |
There was a problem hiding this comment.
Maybe better to implement this as the more generalized concept of a BlockRefRange which would be similar to Blockrange except:
- Both fields required.
- It's based on
BlockRefrather thanBlockNumber
Please feel welcome to ignore if you think not sufficiently valuable.
Goal: Make this a more reusable data model in the case we might have a similar goal somewhere else now / later.
There was a problem hiding this comment.
@lightwalker-eth thank you for suggestion. The current specialized vs generalized balance feels right to me. If we need more generic concepts in the future, we can always create them 👍
Lite PR
Tip: Review docs on the ENSNode PR process
Summary
Indexing Status Buildermodule has been introduced to ENSIndexer. This module aims to host an abstraction layer that primarly relies onviemand@ensnode/ponder-sdkabstractions. It is still using@ensnode/ensnode-sdkfor convenience, but leave the possiblity to iterate and remove@ensnode/ensnode-sdkdependency in the future.Why
OmnichainIndexingStatusSnapshotbased on abstractions fromviemand@ensnode/ponder-sdk(and if really needed, for the time being, from@ensnode/ensnode-sdk).Testing
Notes for Reviewer (Optional)
Pre-Review Checklist (Blocking)